Unintended Consequences of using Tests to Improve Learning: How Improvement-Oriented Resources Heighten Conceptions of Assessment as School Accountability

Over the past decade, the New Zealand government has created a set of resources to support teachers’ use of assessment for learning. These include Assessment Tools for Teaching and Learning (asTTle), a software program enabling teachers to create personalized but standardized tests for diagnostic purposes, and Assess to Learn (AtoL), an intensive professional development program. These resources were expected to increase teacher agreement that improvement is the major purpose of assessment. Instead, a 2008 sample of teachers completing the Teacher Conceptions of Assessment questionnaire showed significantly higher agreement that assessment is about school accountability than participants in previous national surveys. Unlike previous surveys, the correlation between school accountability and improvement conceptions was not statistically significant. However, as only the improvement conception predicted the practices teachers used to define assessment (β = .32), it appears that these teachers still saw many of the practices they used in the classroom (e.g. oral and interactive assessments) as improvement-oriented. Interviews with twenty-six of the surveyed teachers identified that while a few saw the new resources as contributing to improvement and accountability purposes, a larger group failed to make that connection. This second group seemed to be unable to accept that tests, an assessment genre traditionally associated with school and student accountability, could be meaningfully used for improvement at the classroom level. These data show that schools and individuals mediate the implementation of any policy initiative and can therefore cause it to have a range of often unintended consequences. In light of this, the thinking of teachers and other educational stakeholders should be taken into account when enacting policy changes

Mixing interview and questionnaire methods: Practical problems in aligning data

Structured questionnaires and semi-structured interviews are often used in mixed method studies to generate confirmatory results despite differences in methods of data collection, analysis, and interpretation. A review of 19 questionnaire-interview comparison studies found that consensus and consistency statistics were generally weak between methods. Problems in aligning data from the two different methods are illustrated in a questionnaire-interview study of teacher conceptions of assessment. Poor alignment appeared attributable to: differences in data collection procedures, the complexity and instability of the construct being investigated, difficulties in making data comparable, lack of variability in participant responses, greater sensitivity to context and seemingly emotive responses within the interview, possible misinterpretation of some questionnaire prompts, and greater control of content exposure in the questionnaire. Results indicated that if ‘confirmatory’ results are being sought, researchers must create tightly aligned and structured instruments; present the construct in a simple, concrete, and highly contextualised manner; collect the two types of data with a minimal time gap; and estimate agreement between methods using consistency statistics. However, the cost of confirmation through strong alignment may lead to the loss of rich complementary data obtained through allowing each method to be analysed in its own right.Accessed 152,051 times on https://pareonline.net from January 12, 2010 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Independent inventors and inbound open innovation: using a resource-based approach to create a tool for screening inventor approaches in order to facilitate technology in-licensing

Open innovation literature identifies independent inventors as a source of novel external knowledge. This knowledge may be licensed into an organisation in order to supplement internal R&D activity, typically as part of an inbound open innovation strategy. In opening an organisation up to approaches from individuals the capacity of the core team to identify promising licensing opportunities is diminished by the sheer volume and variable quality of approaches received. Based on a survey of 202 UK independent inventors this paper utilises a resource-based approach to identifying the key resources possessed by successful independent inventors. Using this data, we devise a preliminary screening tool to facilitate technology in-licensing from independent inventors

The future of self-assessment in classroom practice: Reframing self-assessment as a core competency

Formative assessment policies argue that student self-assessment of work products and processes is useful for raising academic performance. This view draws on self-regulation of learning theories about setting targets and evaluating progress against criteria as a basis for meta-cognitively informed improvement of learning outcomes. However, the reliability of assessment is necessary for the validity of assessment interpretations. Research into psychological processes underlying the human ability to self-evaluate work raises serious doubts about the quality of students’ judgments. It has been shown that novices (i.e., students) tend to lack sufficient knowledge by which to evaluate their own work appropriately and, when possessing that knowledge, they are likely not to make use of it. Additionally, psychological safety factors indicate that self-assessment can be compromised by interpersonal relations present in classroom environments. Furthermore, since progress in many educational domains is relatively ill-defined, it is difficult for learners, let alone instructors, to validly evaluate progress or status. A recent review of studies in elementary and secondary schooling (K-12) found that the correlation between self-ratings and teacher ratings, between self-estimates of performance and actual test scores, and between student and teacher rubric-based judgments tended to be positive, ranging from weak to moderate (i.e., values ranging from r ≈ .20 to .80), with few studies reporting correlations r > .60. However, these values were not consistent across student experience and academic proficiency, with much less accuracy seen with younger and less proficient students. This paper argues that, instead of treating student self-assessment as an assessment method, a more pedagogical approach is needed that treats self-assessment as an important student competence or skill. As such self-evaluation needs a developmentally appropriate curriculum that is developed incrementally over time through consistent, scaffolded training and practice. Hence, to improve student skills in self-assessment, research must examine how the underlying skills required for self-assessment map onto student cognitive capabilities and the task characteristics of the work students are likely to encounter as they progress through schooling.  This progression (like those developed for other key competencies like literacy, numeracy, and critical thinking) would help teachers consistently implement developmentally appropriate versions of self-assessment, giving students opportunities to develop these life-long skills

Magnetic Structure and Magnetoelectric Coupling in Bulk and Thin Film FeVO\u3csub\u3e4

We have investigated the magnetoelectric and magnetodielectric response in FeVO4, which exhibits a change in magnetic structure coincident with ferroelectric ordering at TN2≈15 K. Using symmetry considerations, we construct a model for the possible magnetoelectric coupling in this system and present a discussion of the allowed spin structures in FeVO4. Based on this model, in which the spontaneous polarization is caused by a trilinear spin-phonon interaction, we experimentally explore the magnetoelectric coupling in FeVO4 thin films through measurements of the electric field-induced shift of the multiferroic phase transition temperature, which exhibits an increase of 0.25 K in an applied field of 4 MV/m. The strong spin-charge coupling in FeVO4 is also reflected in the significant magnetodielectric shift, which is present in the paramagnetic phase due to a quartic spin-phonon interaction and shows a marked enhancement with the onset of magnetic order which we attribute to the trilinear spin-phonon interaction. We observe a clear magnetic field-induced dielectric anomaly at lower temperatures, distinct from the sharp peak associated with the multiferroic transition, which we tentatively assign to a spin-reorientation crossover. We also present a magnetoelectric phase diagram for FeVO4

Spiritual Emergence(y), Psychosis, and Personality: Investigating the Role of Schizotypy

Spiritual emergency (SEY) refers to a process of spiritual emergence (SE) or awakening that becomes traumatic for an individual, leading to a state of psychological crisis. There is evidence that SE(Y) is associated with both psychotic symptomatology and personality traits. This study examined the relationship between SE(Y), psychotic symptoms, and schizotypy, a construct that addresses psychotic-like personality traits in the general population. A total of 250 participants completed an anonymous online questionnaire. Results showed that SE(Y) was positively correlated with positive symptoms of psychosis and schizotypy, but demonstrated only very weak to no correlations with negative symptoms of psychosis and schizotypy. The results also showed that disorganized schizotypy mediated the relationship between positive schizotypy and crisis; and positive schizotypy mediated the relationship between disorganized schizotypy and crisis. The clinical implications of these results include the identification of measurable clinical and personality markers that may help: (1) differentiate between SE(Y) and cases of clinical psychosis that have a poor prognosis, and (2) identify individuals who are at risk of experiencing the potentially debilitating effects of SE(Y)

The Production Manual

207 p. ; col. ill. ; 30 cm

Lessons Learned in Pilot Testing Specialty Consultations to Benefit Individuals with Lower Limb Loss

Telerehabilitation technologies enable the delivery of rehabilitation services from providers to people with disabilities as well as specialty care consultations. This article discusses the barriers experienced when planning and pilot testing a telerehabilitation multi-site specialty consultation for specialists in their medical centers, and the lessons learned. The barriers included integration and participation, coordination across organizational units, and privacy and information security. Lessons learned included the need for collaboration across multiple departments, telerehabilitation equipment back-ups, and anonymous and private communication protocols. Despite delays resulting from coordination at multiple levels of a national organization, we developed a program plan and successfully implemented a pilot test of the southeast region program.  Specialty consultation using telerehabilitation delivery methods requires identifying provider preferences for technological features. Lessons learned could inform development of outpatient telerehabilitation for patients with amputations and studies of patients and providers involved in telerehabilitation

Standardisation of data from real-time quantitative PCR methods – evaluation of outliers and comparison of calibration curves

BACKGROUND: As real-time quantitative PCR (RT-QPCR) is increasingly being relied upon for the enforcement of legislation and regulations dependent upon the trace detection of DNA, focus has increased on the quality issues related to the technique. Recent work has focused on the identification of factors that contribute towards significant measurement uncertainty in the real-time quantitative PCR technique, through investigation of the experimental design and operating procedure. However, measurement uncertainty contributions made during the data analysis procedure have not been studied in detail. This paper presents two additional approaches for standardising data analysis through the novel application of statistical methods to RT-QPCR, in order to minimise potential uncertainty in results. RESULTS: Experimental data was generated in order to develop the two aspects of data handling and analysis that can contribute towards measurement uncertainty in results. This paper describes preliminary aspects in standardising data through the application of statistical techniques to the area of RT-QPCR. The first aspect concerns the statistical identification and subsequent handling of outlying values arising from RT-QPCR, and discusses the implementation of ISO guidelines in relation to acceptance or rejection of outlying values. The second aspect relates to the development of an objective statistical test for the comparison of calibration curves. CONCLUSION: The preliminary statistical tests for outlying values and comparisons between calibration curves can be applied using basic functions found in standard spreadsheet software. These two aspects emphasise that the comparability of results arising from RT-QPCR needs further refinement and development at the data-handling phase. The implementation of standardised approaches to data analysis should further help minimise variation due to subjective judgements. The aspects described in this paper will help contribute towards the development of a set of best practice guidelines regarding standardising handling and interpretation of data arising from RT-QPCR experiments